Repository navigation
Conversation
|
This PR has been marked as stale due to 21 days of inactivity. Please comment or remove the stale label to keep it open. Otherwise, it will be automatically closed in 7 days. |
|
This PR has been marked as stale due to 21 days of inactivity. Please comment or remove the stale label to keep it open. Otherwise, it will be automatically closed in 7 days. |
|
@errose28 can we resurrect this PR? |
|
This PR has been marked as stale due to 21 days of inactivity. Please comment or remove the stale label to keep it open. Otherwise, it will be automatically closed in 7 days. |
|
@errose28 could you please rebase the branch? Its a good change and we should get it merged. |
* master: (892 commits) HDDS-16784. Run rebot in pod in kubernetes check (apache#11938) HDDS-17705. Stop Recon OM sync before closing the metadata manager (apache#11936) HDDS-17755. Use isSupportedBy() for OzoneManagerVersion check (apache#12002) HDDS-17770. Bump zizmorcore/zizmor-action to 0.8.1 (apache#11987) HDDS-16701. S3G: Avoid calling getS3Volume in DELETE_KEY (apache#11740) HDDS-16616. Replace Pair/Triple with domain objects in SCM node decommission code (apache#11984) HDDS-17500. Return x-amz-server-side-encryption on GET and HEAD for encrypted objects (apache#11972) HDDS-17478. Share aws-chunked parsing between SignedChunksInputStream and UnsignedChunksInputStream (apache#11992) HDDS-17410. Record STS revocation cutoff when revoking tenant access id (apache#11796) HDDS-17635. Datanode ReadBlock: a cancelled streaming call is not a container failure (apache#11891) HDDS-17706. Avoid retrying datanode usage query for unknown UUID (apache#11947) HDDS-17407. Allow for clock skew with revocation (apache#11952) HDDS-17421. SCM terminates when it applies Raft log entry written by pre-ZDU SCM finalization (apache#11925) HDDS-16951. Remove the per-call InfoBucket from rooted-ofs rename on FSO buckets (apache#11674) HDDS-17602. Use different runners for some jobs (apache#11977) HDDS-17409. Record STS revocation cutoff when revoking S3 secrets (apache#11794) HDDS-17515. Move KeyLifecycleService properties to a reconfigurable config group (apache#11912) HDDS-17465. Recon keeps the old container mappings of an overwritten key (apache#11762) HDDS-17503. [Recon] Report OM pending deletion as purgeable and snapshot-held size (apache#11855) HDDS-17581. Add TracingUtil span helpers and span-kind aware tracing proxies (apache#11852) ... Conflicts: hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/TestTrashOzoneFileSystem.java hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/service/TestDirectoryDeletingService.java hadoop-ozone/ozone-manager/src/test/java/org/apache/hadoop/ozone/om/service/TestQuotaRepairTask.java
|
Just did the reverse merge from master. I skimmed it and it looks correct, although it's been a while since I looked at this. If more updates are needed I may not have time to get to them until the remaining ZDU bugs are resolved. |
ptlrs
left a comment
ptlrs
left a comment
There was a problem hiding this comment.
Thanks for the PR @errose28.
I had taken a look at this PR a while back and at that time one my concerns was that some of the existing metrics/dashboards may break. From the user's perspective, it may appear that the number of operations like deletion are now higher post this PR than what they may have triggered themselves.
It would be good to have some data about what the difference would look like and if it is acceptable. Otherwise system generated operations should be tagged in a way that that they can be isolated.
What changes were proposed in this pull request?
The OM's
ProtocolMessageMetricsis only used in the server side translator. This captures end-to-end latency for all client facing requests (likeRenameKey), but not for internal OM requests submitted directly to Ratis by background services (likePurgeKeys). This causes latency metrics for internal requests to always be 0.This PR wires the metrics to
OzoneManagerRatisUtils#submitRequestas well to capture latency of requests submitted directly to the Ratis server.What is the link to the Apache JIRA
HDDS-16238
How was this patch tested?
OzoneManagerRatisUtilsand all existing background services.